1. Field of the Invention
This invention relates to the field of video compression, and in particular to a video decoder which provides fully IEEE compliant MPEG decoding with a reduced complexity architecture.
2. Description of the Related Art
Full-motion digital video requires a large amount of storage and data transfer bandwidth. Thus, video systems use various types of video compression algorithms to reduce the amount of necessary storage and transfer bandwidth. In general, different video compression methods exist for still graphic images and for full-motion video. Intraframe compression methods are used to compress data within a still image or single frame using spatial redundancies within the frame. Interframe compression methods are used to compress multiple frames, i.e., motion video, using the temporal redundancy between the frames. Interframe compression methods are used exclusively for motion video, either alone or in conjunction with intraframe compression methods.
Intraframe or still image compression techniques generally use frequency domain techniques, such as the discrete cosine transform (DCT). Intraframe compression typically uses the frequency characteristics of a picture frame to efficiently encode a frame and remove spatial redundancy. Examples of video data compression for still graphic images are JPEG (Joint Photographic Experts Group) compression and RLE (run-length encoding). JPEG compression is a group of related standards that use the discrete cosine transform (DCT) to provide either lossless (no image quality degradation) or lossy (from imperceptible to severe image degradation) compression. Although JPEG compression was originally designed for the compression of still images rather than video, JPEG compression is used in some motion video applications. The RLE compression method operates by testing for duplicated pixels in a single line of the bit map and storing the number of consecutive duplicate pixels rather than the data for the pixels themselves.
In contrast to compression algorithms for still images, most video compression algorithms are designed to compress full motion video. As mentioned above, video compression algorithms for motion video use a concept referred to as interframe compression to remove temporal redundancies between frames. Interframe compression involves storing only the differences between successive frames in the data file. Interframe compression stores the entire image of a reference frame or anchor frame, generally in a moderately compressed format. Successive frames are compared with the anchor frame, and only the differences between the anchor frame and the successive frames are stored. Periodically, such as when new scenes are displayed, new anchor frames are stored, and subsequent comparisons begin from this new reference point. It is noted that the interframe compression ratio may be kept constant while varying the video quality. Alternatively, interframe compression ratios may be content-dependent, i.e. if the video clip being compressed includes many abrupt scene transitions from one image to another, the compression is less efficient. Examples of video compression which use an interframe compression technique are MPEG, DVI and Indeo, among others.
The compression standard referred to as MPEG (Moving Pictures Experts Group) compression is a set of methods for compression and decompression of full motion video images which uses the interframe and intraframe compression techniques described above. MPEG compression uses both motion compensation and discrete cosine transform (DCT) processes, among others, and can yield very high compression ratios.
The two predominant MPEG standards are referred to as MPEG-1 and MPEG-2. The MPEG-1 standard generally concerns inter-field data reduction using block-based motion compensation prediction, which generally uses temporal differential pulse code modulation. The MPEG-2 standard is similar to the MPEG-1 standard, but includes extensions to cover a wider range of applications, including interlaced digital video such as high definition television (HDTV).
Interframe compression methods such as MPEG are based on the fact that, in most video sequences, the background remains relatively stable while action takes place in the foreground. The background may move, but large portions of successive frames in a video sequence are redundant. MPEG compression uses this inherent redundancy to encode or compress frames in the sequence.
An MPEG stream includes three types of pictures, referred to as the Intra (I) frame, the Predicted (P) frame, and the Bi-directional Interpolated (B) frame. The I (intra) frames contain the video data for the entire frame of video and are typically placed every 10 to 15 frames. Intraframes provide entry points into the file for random access, and are generally only moderately compressed. Predicted frames are encoded with reference to a past frame, i.e., a prior Intraframe or Predicted frame. Thus P frames only include changes relative to prior I or P frames. In general, P frames receive a fairly high amount of compression and are used as references for future P frames. Thus, both I and P frames are used as references for subsequent frames. Bi-directional pictures include the greatest amount of compression and require both a past and a future reference in order to be encoded. Bi-directional frames are never used as references for other frames.
In general, for the frame(s) following a reference frame, i.e., P and B frames that follow a reference I or P frame, only small portions of these frames are different from the corresponding portions of the respective reference frame. Thus, for these frames, only the differences are captured, compressed and stored. The differences between these frames are typically generated using motion vector estimation logic, as discussed below.
When an MPEG encoder receives a video file or bitstream, the MPEG encoder generally first creates the I frames. The MPEG encoder may compress the I frame using an intraframe compression technique. After the I frames have been created, the MPEG encoder divides following frames into a grid of 16.times.16 pixel squares called macroblocks. For a following target frame, i.e., the frame being encoded, the encoder searches for a best fit or best match between the target picture macroblock and a region in a neighboring frame, referred to as an anchor frame. For a target P frame, the encoder searches in a prior I or P frame. For a target B frame, the encoder searches in a prior and a subsequent I or P frame. When a best match is found, the encoder transmits a vector movement code or motion vector. The vector movement code or motion vector includes a pointer to the best fit region in the anchor frame as well as information on the difference between the best fit region and the respective target block. The blocks in target pictures that have no change relative to the corresponding region in the reference or search frame are ignored. Thus the amount of data that is actually stored for these frames is significantly reduced.
After motion vectors have been generated, the encoder then encodes the changes using spatial redundancy. Thus, after finding the changes in location of the macroblocks, the MPEG algorithm further calculates and encodes the difference between the macroblock and corresponding best-fit region. Encoding the difference is accomplished through a math process referred to as the discrete cosine transform or DCT. This process divides the macroblock into four sub-blocks, seeking out changes in color and brightness. Human perception is more sensitive to brightness changes than color changes. Thus the MPEG algorithm devotes more effort to reducing color space rather than brightness.
Therefore, MPEG compression is based on two types of redundancies in video sequences, these being spatial, which is the redundancy in an individual frame, and temporal, which is the redundancy between consecutive frames. Spatial compression is achieved by considering the frequency characteristics of a picture frame. Each frame is divided into non-overlapping blocks and respective sub-blocks, and each block is transformed via the discrete cosine transform (DCT).
After the transformed blocks are converted to the "DCT domain", each entry (i.e. DCT coefficient) in the transformed block is quantized with respect to a set of quantization tables. The quantization step for each entry can vary, taking into account the sensitivity of the human visual system (HVS) to the frequency. Since the HVS is more sensitive to low frequencies, most of the high frequency entries are quantized to zero. In this step where the DCT coefficients are quantized, information is lost and errors are introduced to the reconstructed image. Run-level encoding will be used to encode the quantized DCT coefficients. This statistical encoding of the consecutive zeroed-valued coefficients corresponding to the higher-order coefficients accounts for considerable compression gain.
For I-frames, the video encoder then performs variable-length coding (VLC) on the DC data. VLC is a reversible procedure for coding data that assigns shorter code words to frequent events and longer code words to less frequent events, thereby achieving additional video compression. Huffman encoding is a particularly well-known form of VLC that reduces the number of bits necessary to represent a data set without losing any information.
For all frames, the coefficient sequence is often organized in a specified orientation termed zigzag ordering to cluster non-zero coefficients early in the series and to encode as many zero coefficients as possible following the last non-zero coefficient in the ordering. Zigzag ordering concentrates the highest spatial frequencies at the end of the series. Once the zigzag ordering has been performed, the encoder performs "run-level coding" on the AC coefficients of I-frames, and both the DC and AC coefficients for P and B-frames. Run-level coding encodes the number of zeros between non-zero coefficients (i.e. the zero run length) and the value of the following non-zero coefficient (i.e. the quantization level) into a single variable-length code word. This process reduces each 8 by 8 block of DCT coefficients to a number of events represented by a non-zero coefficient and the number of preceding zero coefficients. Because the high-frequency coefficients are more likely to be zero, zigzag ordering prior to run-level coding results in additional video compression.
The final compressed video data is then ready to be provided to a storage device or transmitted over a transmission medium for reception and decompression by a remotely located decoder. Because of the picture dependencies, i.e., the temporal compression, the order in which the frames are transmitted, stored, or retrieved, is not necessarily the display order, but rather an order required by the decoder to properly decode the pictures in the bitstream. For example, a typical sequence of frames, in display order, might be shown as follows:
I B B P B B P B B P B B I B B P B B P 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
By contrast, the bitstream order corresponding to the given display order would be as follows:
 I P B B P B B P B B I B B P B B P B B 0 3 1 2 6 4 5 9 7 8 12 10 11 15 13 14 18 16 17
Because the B frame depends on a subsequent I or P frame in display order, the I or P frame must be transmitted and decoded before the dependent B frame.
As discussed above, temporal compression makes use of the fact that most of the objects remain the same between consecutive picture frames, and the difference between objects or blocks in successive frames is their position in the frame as a result of motion (either due to object motion, camera motion or both). The key to this relative encoding is motion estimation. In general, motion estimation is an essential processing requirement in most video compression algorithms. In general, motion estimation is the task of identifying temporal redundancy between frames of the video sequence.
The video decoding process is generally the inverse of the video encoding process and is employed to reconstruct a motion picture sequence from a compressed and encoded bitstream. The data in the bitstream is decoded according to a syntax that is defined by the data compression algorithm. The decoder must first identify the beginning of a coded picture, identify the type of picture, then decode each individual macroblock within a particular picture.
When encoded video data is transferred to a video decoder, the encoded video data is received and stored in a channel buffer. The data is then retrieved from the channel buffer by a decoder or reconstruction device for performing the decoding process. When the MPEG decoder receives the encoded stream, the MPEG decoder reverses the above operations. Thus the MPEG decoder performs inverse scanning to remove the zigzag ordering, inverse quantization to de-quantize the data, and the inverse DCT to convert the data from the frequency domain back to the pixel domain. The MPEG decoder also performs motion compensation using the transmitted motion vectors to re-create the temporally compressed frames.
When frames are received which are used as references for other frames, such as I or P frames, these frames are decoded and stored in memory. When a reconstructed frame is a reference or anchor frame, such as an I or a P frame, the reconstructed frame replaces the oldest stored anchor frame and is used as the new anchor for subsequent frames.
When a temporally compressed or encoded frame is received, such as a P or B frame, motion compensation is performed on the frame using the neighboring decoded I or P reference frames, also called anchor frames. The temporally compressed or encoded frame, referred to as a target frame, will include motion vectors which reference blocks in neighboring decoded I or P frames stored in the memory. The MPEG decoder examines the motion vector, determines the respective reference block in the reference frame, and accesses the reference block pointed to by the motion vector from the memory.
In order to reconstruct a B frame, the two related anchor frames or reference frames must be decoded and available in a memory, referred to as the picture buffer. This is necessary since the B frame was encoded relative to these two anchor frames. Thus the B frame must be interpolated or reconstructed using both anchor frames during the reconstruction process.
After all of the macroblocks have been processed by the decoder, the picture reconstruction is complete. The decoded frame is stored into a frame store buffer. In some configurations, the channel and frame store buffers are incorporated into a single integrated memory buffer. A display processor retrieves the picture data for display by an appropriate display device, such as a TV monitor or the like.
The MPEG standard does not specify implementations for encoders and decoders. The system designer is free to choose among many DCT algorithms and implementations, most of which provide significant savings in computational effort by taking advantages of symmetries in the DCT. Although the various methods theoretically yield the same end result, it is often necessary to implement the methods with extended bit lengths to keep truncation errors from propagating and causing significant output errors. In IEEE Standard 1180-1990, a standard is provided for maximum allowable output error from inverse DCT implementations. It is desirable to provide video decoders with minimal implementation complexity and cost which maintain compliance with this standard.